Improving Generalisation for Temporal Difference Learning: The Successor Representation
نویسنده
چکیده
Estimation of returns over time, the focus of temporal difference (TD) algorithms, imposes particular constraints on good function approximators or representations. Appropriate generalisation between states is determined by how similar their successors are, and representations should follow suit. This paper shows howTDmachinery can be used to learn such representations, and illustrates, using a navigation task, the appropriately distributed nature of the result.
منابع مشابه
Running head: SUCCESSOR REPRESENTATION and TEMPORAL CONTEXT The Successor Representation and Temporal Context
The successor representation was introduced into reinforcement learning by Dayan (1993) as a means of facilitating generalization between states with similar successors. Although reinforcement learning in general has been used extensively as a model of psychological and neural processes, the psychological validity of the successor representation has yet to be explored. An interesting possibilit...
متن کاملImproving Generalization for Temporal Difference Learning: The Successor Representation
Estimation of returns over time, the focus of temporal difference (TD) algorithms, imposes particular constraints on good function approximators or representations. Appropriate generalization between states is determined by how similar their successors are, and representations should follow suit. This paper shows how TD machinery can be used to learn such representations, and illustrates, using...
متن کاملThe Successor Representation and Temporal Context
The successor representation was introduced into reinforcement learning by Dayan ( 1993 ) as a means of facilitating generalization between states with similar successors. Although reinforcement learning in general has been used extensively as a model of psychological and neural processes, the psychological validity of the successor representation has yet to be explored. An interesting possibil...
متن کاملFactored Temporal Difference Learning in the New Ties Environment
Although reinforcement learning is a popular method for training an agent for decision making based on rewards, well studied tabular methods are not applicable for large, realistic problems. In this paper, we experiment with a factored version of temporal difference learning, which boils down to a linear function approximation scheme utilising natural features coming from the structure of the t...
متن کاملAdvantages and Limitations of using Successor Features for Transfer in Reinforcement Learning
One question central to Reinforcement Learning is how to learn a feature representation that supports algorithm scaling and re-use of learned information from different tasks. Successor Features approach this problem by learning a feature representation that satisfies a temporal constraint. We present an implementation of an approach that decouples the feature representation from the reward fun...
متن کامل